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I.  INTRODUCTION 

The  Lagrange  multiplier  test(LM),  the  likelihood  ratio  test.(LR), 
and  the  Wald  test  (W) ,  are  frequently  proposed  as  alternative  means 
for  testing  parametric  restrictions  in  a  linear  regression  model. 

Since  they  all  converge  to  the  same  limiting  Chi-square  distribution 
they  have  asymptotically  the  same  power  characteristics.  Consequently 
the  three  tests  are  considered  equivalent  alternatives  for  large 
samples,  and  the  choice  among  them  is  usually  made  on  convenience 
grounds  (e.g. ,  ease  of  estimation). 

However,  Savin  (1976)  and  Bemdt  and  Savin  (1977),  among  others, 
have  shown  that  the  numerical  value  of  the  IM  test  statistic  is  always 
less  than  that  of  LR,  which  in  turn  is  less  than  that  of  W.  This 
implies  that  if  we  use  the  same  critical  value,  as  seems  indicated 
by  the  fact  that  all  three  tests  converge  to  the  same  limiting  Chi- 
square  distribution,  the  Wald  test  will  reject  the  null  hypothesis 
most  often,  while  the  LM  test  will  reject  it  least  often.  This 
difference  in  rejection  probabilities  raises  the  possibility  of 
conflicting  conclusions  from  the  three  tests. 

'>In  this  paper  we  show  that  the  three  tests  are  monotonic  functions 
of  each  other.  This  implies  that  they  have  the  same  power  characteristics. 
If  the  critical  values  are  specified  such  as  to  equate  the  probabilities 
of  type  I  error,  the  probabilities  of  type  II  error  will  be  equal  as 
well,  and  conflicting  results  are  impossible. 

The  critical  values  of  the  three  tests  are  related  to  each  other 
by  the  same  monotonic  functions  as  the  three  test  statistics.  The 
"conflicts’*  pointed  out  in  the  literature  arise  only  when  the  exact 
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v-  critical  values  for  each  test  are  replaced  by  the  asymptotically 
justified  critical  value  of  a  Chi-square  distribution.  The  extent 
to  which  this  critical  value  differs  from  the  exact  one  varies  from 
test  to  test. 

Determining  the  exact  critical  values  for  finite  samples  is 
somewhat  difficult,  because  the  exact  finite  sample  distributions 
of  the  LM  and  LR  test  are  generally  not  known.  However,  under  the 
assumption  of  normality,  the  W  test  can  be  transformed  into  an 
F-test  by  applying  the  standard  degrees  of  freedom  correction. 
Furthermore,  since  the  LR  and  LM  tests  are  functionally  related 
to  the  W  test,  they  can  be  transformed  into  the  same  test  statistic. 
We  can  thus  conduct  an  exact  finite  sample  test,  based  either  on  the 
LM,  LR  or  W  statistic  without  running  the  risk  of  obtaining 
conflicting  results.^ 


2.  THE  FORM  OF  THE  TESTS 

Consider  the  model: 

(1) 

(2) 

where  X  is  of  rank  k. 

*  ^  2 

Let  P  and  a  be  the  maximum- likelihood  estimators  obtained  by 
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unconstrained  maximization  of  the  likelihood  function,  and  3  and  o 
the  estimators  obtained  with  the  constraint  r  =  Rg  imposed.  Further¬ 
more,  we  shall  need  an  estimator  for  the  Lagrange  multiplier  (y)  and 

the  ratio  of  the  unconstrained  to  constrained  maxima  of  the  likelihood 
function  (X).  The  three  test  statistics  can  be  written  as: 

(3)  LR  =  -2  log  X 

(4)  W  -  [(r  -  R6)’(Ra’X)"1R,)'1(r  -  R8)]/52 

(5)  LM  -  [u(R(X,X)"1R')ii]/o2 

where  R  is  the  matrix  of  constraints  and  is  of  rank  q. 

To  simplify  the  notation  let  A  “  R(l/n  X’X)  *R' .  From  the  first- 
order  conditions  for  maximizing  the  likelihood  function  subject  to 
the  constraint  we  can  obtain  an  expression  for  y: 


Y  =  X6  +  e 
e  ^  N(0,o2I) 
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We  can  now  rewrite  Eqs.  (4)  and  (5)  as 

(7)  W  =  n[(r  -  R|3)’A-1(r  -  R8)]/o2 

(8)  LM  =  n [ (r  -  R8)’A_1(r  -  R6)]/52  . 

This  implies  that  we  can  express  the  LM  test  as 

-2 

(9)  LM  =  W  . 

c 

If  we  make  use  of  the  fact  that  the  likelihood  ratio 
A  2  „2  n  '2 

X  ■  (a  /a  )  we  can  express  Eq.  (9)  as 

(10)  LM  -  X2/n  •  W  . 

Equation  (10)  makes  it  obvious  that  the  LM  and  W  test  statistics  are 
asymptotically  equivalent. 

To  express  LR  as  a  function  of  W  consider  cr 2  =  (Y  -  X6)  ’  (Y  -  X8)/n 
and  8*8+  n(X,X)_1R' A_1(r  -  RB).  Then,  letting  e  =  Y  -  X8,  we  get 
by  substitution: 

a2  -  l/n(e’e  -  2e,X(l/nX’X)"1R'A"1(r  -  RB) 

(11)  +  (r  -  R8)'A-1n[R(X,X)"1X,X(X,X)"1R,]A"1(r  -  R8)> 

*  62  +  (r  -  R8) ’ A~^(r  -  RB)  . 

We  can,  therefore,  express  Eq.  (7)  as 


(12) 


W  ■  n(o2  -  a2)/a2 


n(oz/d"  -  1) 


.'•"i 


-5- 


which  is  equivalent  to 

(13)  W  =  n(X~2/n  -  1)  . 

Combining  Eqs.  (13)  and  (10)  leads  to 

(14)  LM  *  n(l  -  X2/n)  . 

Let  0  =  X  2^n  which  is  larger  or  equal  to  unity  with  probability  one. 
Then,  from  Eqs.  (3),  (13),  and  (14)  the  three  tests  can  be  rewritten 
as 

(15)  LR  *  n  log  0 

(16)  W  «  n(0  -  1) 

(17)  LM  -  n(l  -  1/0)  . 

Thus,  since  (1  -  1/0)  <  log  0  <  0  -  1  we  have  the  well-known 
inequality  relationship 


(18) 


LM  <  LR  <  W  . 
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3.  THE  THREE  TESTS  AS  TRANSFORMATIONS  OF  THE  F-TEST* 


If  we  divide  Eq.  (4)  by  q  we  get 

(19)  W/q  =  [(r  -  RB)'  (R(X,X)"1R')“1(r  -  RB)]/S2  •  q 

which  is  distributed  as  F(q,n  -  k) .  From  EqS.  (15)  and  (16)  we  can 
express  W/q  as  a  function  of  LR,  i.e. , 

(20)  W/q  =  n  •  [exp(LR/n)  -  l]/q 

which  also  follows  an  F(q,n  -  k)  distribution.  With  Eqs.  (19)  and  (20) 
we  can  thus  transform  either  W  or  LR  into  an  F  statistic  of  which  we 
know  the  exact  finite  sample  distribution.  This  allows  us  to  conduct 
exact  finite  sample  tests. 

Much  of  the  justification  of  the  LM  test  is  based  on  the  fact 
that  it  relies  exclusively  on  parameter  estimates  from  the  restricted 
model.  That  makes  it  particularly  attractive  in  situations  where  the 
unrestricted  model  is  difficult  or  impossible  to  estimate.  In  the 
previous  section  we  have  given  a  formula  (Eq.  (9))  by  which  we  could 

compute  the  W  test  statistic  from  the  LM  statistic.  But  it  relies 

-2 

on  the  unconstrained  estimate  c  which  defeats  the  major  advantage 
of  the  LM  test.  We  have  to  find  an  alternative  way  of  calculating 
an  F  statistic  from  the  LM  statistic. 

Recall  that  the  LM  test  is  based  on  maximizing  the  likelihood 
function,  subject  to  the  constraint,  by  the  Lagrange  method  and  then 

*Throughout  this  section  we  assume  that  the  F-test  is  defined. 

In  particular  we  require  that  n  -  k  >  0. 
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testing  how  close  the  estimated  Lagrange  multipliers  (u)  are  to  zero. 
We  have  shown  that  under  the  assumptions  of  the  linear  model  this  is 
equivalent  to  testing 


(21)  LM  =  n[(r  -  RB) ’A_1(r  -  R8)]/52  . 


But  note  that  we  can  express  o  as  a  function  of  the  unconstrained 
parameters.  In  particular  we  have  (see  Eq.  (11)) 


a2  =  a2  +  (r  -  R8)'A-1(r  -  RB) 


so  that  we  can  express  the  LM  statistic  as 


(22)  LM  =  n  •  [ (r  -  RB)'A-1(r  -  RB)]/[o2  +  (r  -  RB) ’ A  X(r  -  RB) ] 


Dividing  LM  by  n  and  inverting  results  in 


(23)  7^  =  1  +  - r-~ - 

^  (r  -  RB) ’A  1(r  -  RB) 


which  can  be  transformed  into 


[iM 


1  (r  -  R  _)  'A_1(r  -  RB)  W 


For  finite  samples  we  can  use  the  statistic 


a  r_n  .  if1  =  iiJL 

q  LLM  AJ 


'2 

a  *  q 


which  follows  an  F  distribution  with  (q,  n  -  k)  degrees  of  freedom. 
Like  the  W  and  LR  test  statistics,  the  IM  statistic  can  thus  be 
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adjasted  for  sample  size  and  degrees  of  freedom  to  result  in  a  test 
statistic  applicable  for  finite  samples.  As  it  does  not  require 
estimating  the  unconstrained  model,  it  is  particularly  useful  in 
situations  where  only  the  constrained  version  can  readily  be  esti¬ 
mated. 
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tangent  (a)  =  sine  (a)/cosine  (a)  which,  if  raised  to  the  power  two, 
corresponds  to  Eq.  (10). 

Continuing  with  the  trigonometric  analogy,  we  see  that  Eq.  (11) 
corresponds  to  the  theorem  of  Pythagoras.  Thus,  if  we  have  an  estimate 
of  any  two  of  the  three  sides  of  the  triangle  in  Fig.  1,  either  from 
the  unconstrained  regression  (o,B)  or  from  the  constrained  one  (o,y), 
we  can  calculate  the  third  side  and  then  conduct  any  one  of  the  three 
tests. 

Note  that  0  and  (r  -  R'C)  are  orthogonal  to  each  other.  In 

statistical  terms  this  means  that  they  are  independently  distributed. 

We  can  thus  compute  the  exact  small  sample  distribution  of 
2 

n  .  [tangent  (a)]  ,  which  turns  out  to  be  a  multiple  of  an  F 
distribution.  For  the  other  two  tests  this  is  not  directly  possible, 
because  the  components  going  into  the  calculation  of  sine  (a)  and 
cosine  (a)  are  not  independent  of  each  other.  However,  by  expressing 
these  tests  as  deterministic  functions  of  tangent  (a),  e.g. ,  through 
the  trigonometric  relations  described  above,  we  can  obtain  test 
statistics  with  known  finite  sample  distributions  as  well. 
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